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Abstract 

How  do  teams  cope  with  a  misalignment  between  their  organizational  structures  and  mission 
requirements?  Through  a  series  of  empirical  and  modeling  efforts,  our  long-term  goal  is  to 
understand  the  nature  of  organizational  congruence,  and  to  predict  and  measure  the  effects  of 
a  lack  of  congruence  on  the  behaviors  of  organizations  as  they  strive  to  adapt.  In  this  paper, 
we  present  a  series  of  lessons  learned  based  on  a  pilot  experiment  in  which  we  explored  the 
performance  of  two  organizations  (functional  and  divisional)  performing  a  single  mission, 
which  in  this  case,  was  not  designed  to  favor  a  particular  organization.  The  lessons  learned 
here  suggest  a  variety  of  empirical  strategies  to  further  our  investigations,  including  measures 
of  performance,  rigorous  attention  to  team  and  mission  definitions,  and  precise  planning  of 
training  structure.  Our  findings  suggest  that  each  of  these  factors  has  the  potential  to 
profoundly  influence  the  results,  reflecting  the  multiple  factors  that  shape  team  performance. 

1.  Introduction 

For  the  past  several  years,  the  Adaptive  Architectures  for  Command  and  Control  (A2C2) 
research  team  has  examined  the  concept  of  organizational  “congruence.”  This  theory 
proposes  that  organizational  effectiveness  can  be  mediated  when  characteristics  like 
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organizational  structure  are  designed  to  fit  the  requirements  of  a  mission  or  environment 
(e.g.,  Donaldson,  2001;  Van  de  Ven  &  Drazin,  1985).  This  view  suggests  that  when 
organizational  structure  is  out  of  “alignment,”  quality  of  performance  should  be  reduced,  and 
if  significant,  this  misalignment  may  drive  organizational  adaptation. 

Some  previous  A2C2  efforts  have  examined  ways  to  measure  “fit”  or  “alignment”  between 
structure  and  task  requirements  (e.g.,  Levchuk,  Meirina,  Levchuk,  Pattipati,  &  Kleinman, 
2001).  Building  on  this  research,  our  goal  is  to  understand  the  processes  that  underlie 
performance  as  teams  attempt  to  cope  with  and  adapt  to  misalignment.  Toward  this  end,  the 
research  reported  here  focuses  on  a  pilot  experiment  designed  to  explore  the  factors  that 
affect  the  performance  of  teams  operating  in  different  structural  organizations.  We  present  a 
series  of  lessons  learned  that  offer  insight  into  the  methodological  implications  of  the 
processes  at  work  that  affect  the  investigation  of  organizational  congruence  and  adaptation. 

1.1  Toward  Empirical  Evaluations  of  Congruence  and  Adaptation 

Over  the  next  year,  the  A2C2  research  team  will  conduct  an  experiment  designed  to  explore 
the  processes  that  characterize  performance  when  teams  are  faced  with  misalignment 
between  mission  environments  and  organizational  structures.  More  specifically,  the  focus  of 
the  experiment  will  be  on  the  use  of  several  modeling  techniques  (e.g.,  Carley  &  Lee,  1998; 
Handley,  Zaidi,  &  Levis,  1999;  Levchuk,  Pattipati,  &  Kleinman,  1998)  to  understand  the 
nature  of  the  congruence  or  “fit”  between  an  organizational  structure  and  mission,  and  to 
predict  and  measure  the  effects  of  a  lack  of  congruence  on  the  behaviors  of  the  organization 
as  it  carries  out  its  mission.  The  reasoning  behind  this  approach  is  that  when  the  structure  of 
an  organization  becomes  too  “incongruent”  with  its  mission,  the  organization  will  “feel  the 
pain”  and  may  (or  may  not)  be  motivated  to  adapt.  By  identifying  the  behaviors  of 
organizations  coping  with  incongruence,  we  hope  to  understand  how  they  function  so  that  we 
can  subsequently  induce  on-line  structural  adaptation  (changes  in  team  organization). 
Consequently,  the  ultimate  goal  of  the  upcoming  empirical  work  is  an  improved 
understanding  of  what  incongruence  looks  like  “in  action”  so  that  we  can  properly  study 
whether  and  how  incongruence  leads  to  structural  adaptation. 

Our  approach  is  to  define  two  disparate  organizational  structures  and  design  two  missions 
(scenarios)  that  exploit  the  differences  in  these  two  structures.  Thus,  the  first  mission 
scenario  will  be  “tuned”  to  organization  1  through  a  high  degree  of  congruence,  and  will  also 
be  “mismatched”  (i.e.,  exhibit  low  congruence)  with  organization  2.  The  reverse  will  be 
necessary  for  the  second  scenario.  Building  on  previous  work  (Hutchins,  Kleinman, 
Hocevar,  Kemple,  &  Porter,  2001),  the  two  organizational  structures  that  we  will  explore  are 
commonly  referred  to  as  functional  and  divisional.  In  this  work,  the  functional  structure  will 
be  organized  such  that  each  participant  will  specialize  in  one  aspect  of  the  mission  such  as 
strike  using  assets  that  are  distributed  across  multiple  platforms  (ships).  In  contrast,  in  the 
divisional  structure,  each  participant  will  have  control  over  a  multifunctional  platform  that  to 
a  large  extent  will  be  able  to  process  a  variety  of  functional  tasks  such  as  air,  surface,  and 
subsurface  threats.  The  two  mission  scenarios  will  be  designed,  theoretically,  to  favor  either 
the  divisional  or  functional  architectures. 


However,  before  directly  embarking  on  this  experiment,  we  conducted  a  pilot  experiment 
designed  to  better  elucidate  the  properties  of  the  missions  and  organizations  necessary  to 
meet  our  goals.  In  short,  given  the  web  of  variables  that  influence  team  performance,  the 
nature  of  the  contextual  factors  that  might  affect  congruence  and  adaptation  is  not  obvious. 
This  means  that  definition  of  the  organizations  and  the  missions  is  non-trivial,  as  are  the 
methodological  subtleties  that  could  affect  the  results.  Accordingly,  the  focus  of  this  paper  is 
on  a  pilot  experiment  in  which  we  explored  the  performance  of  two  organizations  (functional 
and  divisional)  performing  a  single  mission,  which  in  this  case,  was  not  designed  a  priori  to 
favor  a  particular  organization.  Below  we  outline  the  design  of  this  pilot  experiment  and  then 
present  the  results  from  this  experiment  in  the  context  of  lessons  learned  that  will  inform  the 
design  of  subsequent  work  on  organizational  congruence  and  adaptation  both  within  and 
beyond  the  A2C2  program. 

2.  Method 

2.1  Participants 

Forty-two  officers  attending  the  Naval  Postgraduate  School  in  Monterey,  CA,  served  as 
participants.  Most  of  the  officers  were  03  or  04  and  several  services  as  well  as  foreign 
militaries  were  represented.  Participants  were  organized  into  seven  teams  of  six  individuals 
each.  Informed  consent  was  obtained  for  all  participants. 

2.2  Apparatus 

Participants  engaged  in  a  simulation  of  a  mission  involving  six  geographically  dispersed 
commanders  in  control  of  assets  including  six  major  platforms:  An  aircraft  carrier,  a  guided 
missile  cruiser,  two  guided  missile  destroyers,  a  guided  missile  frigate,  and  an  attack 
submarine.  The  platforms  each  had  a  variety  of  weapons  with  specified  capabilities,  and 
some  had  assets  such  as  aircraft  and  helicopters.  The  simulation  was  implemented  in  the 
Distributed  Dynamic  Decision-making  (DDD)  environment.  The  DDD  is  a  distributed  real¬ 
time  simulation  environment  that  supports  a  variety  synthetic  team  tasks.  An  example  of  the 
simulation  display  is  shown  in  Figure  1 . 

2.3  Procedure 

Participants  began  the  experiment  by  learning  how  to  control  the  various  assets  and  how  to 
use  the  various  functions  imbedded  in  the  DDD  simulation.  This  “buttonology”  training 
lasted  for  approximately  two  hours  (including  simulation  time  and  briefing  time).  Next,  they 
conducted  training  in  one  organizational  structure  (functional  or  divisional)  on  variants  of  the 
mission  scenario.  This  training  lasted  for  approximately  60  minutes  and  was  followed  by  an 
approximately  40-minute  testing  period  in  the  actual  mission  scenario  (the  test  period  lasted  a 
total  of  two  hours  including  briefing  and  debriefing).  The  participants  then  came  back  a  few 
days  later  to  conduct  training  in  the  second  organizational  structure  (60  minutes)  followed  by 
testing  on  the  actual  mission  scenario  (approximately  40  minutes  in  the  scenario,  two  hours 
total).  Thus,  organizational  structure  was  manipulated  as  a  repeated  measures  factor  that  was 


counterbalanced  across  teams:  Half  of  the  teams  started  in  divisional  followed  by  functional 
and  half  started  in  functional  followed  by  divisional. 


Figure  1.  The  DDD  simulation  display.  The  white  areas  represent  water,  the  colored  areas  represent  land 
(friendly  areas  outlined  in  bold),  and  the  various  icons  represent  friendly  and  hostile  assets  of  different  types. 

More  specifically,  in  the  functional  organizational  structure,  a  participant  charged  with  one 
aspect  of  the  mission,  such  as  air  warfare,  “owned”  (had  direct  control  of)  the  assets 
necessary  to  do  that  part  of  the  mission  even  if  those  assets  were  on  various  platforms.  Each 
of  the  six  players  had  primary  control  of  one  of  the  following  areas:  strike,  air  warfare, 
ballistic  missile  defense,  surveillance/search  and  rescue,  surface  warfare  or  undersea  warfare. 
In  contrast,  in  the  divisional  organizational  structure,  players  had  direct  control  over  all  assets 
on  one  multifunctional  platform:  A  carrier  (strike,  air  warfare,  search  and  rescue),  a  guided 
missile  cruiser  (strike,  ballistic  missile  defense,  surveillance,  surface  warfare,  undersea 
warfare),  a  guided  missile  destroyer  (strike,  ballistic  missile  defense,  surveillance,  surface 
warfare),  a  second  guided  missile  destroyer  (Strike,  ballistic  missile  defense,  surveillance, 
search  and  rescue,  surface  warfare),  a  guided  missile  frigate  (surveillance,  surface  warfare, 
undersea  warfare,  search  and  rescue),  and  an  attack  submarine  (strike,  undersea  warfare). 
Each  of  the  platforms  were  located  in  a  geographically  distinct  region,  with  some  overlap  of 
range  such  that  multiple  players  could  access  the  same  targets  in  certain  areas. 

The  mission  involved  preparing  the  battlespace  for  the  arrival  of  follow-on  forces  by 
engaging  hostile  assets  perceived  as  immediate  threats  (aircraft,  patrol  boats,  destroyers,  and 
submarines).  Simultaneously,  players  were  charged  with  destroying  an  enemy  air  base  and 


an  enemy  naval  base  that  were  heavily  guarded  by  surface-to-air  missile  (SAM)  sites.  In 
addition,  players  had  to  defend  neighboring  foreign  friendly  areas  from  SCUD  missile 
attacks.  Critically,  the  mission  environment  was  significantly  characterized  by  a  variety  of 
time-critical  tasks  (e.g.,  SCUD  missile  launchers,  coastal  defense  launchers,  search  and 
rescue)  that  required  immediate  response.  Likewise,  many  of  these  tasks  required 
coordination  between  assets  (e.g.,  a  surveillance  sensor  to  locate  a  target  and  an  asset  with 
which  to  destroy  it)  that  could  be  controlled  by  one  or  more  players.  Note  that  the  area  of 
operation  also  contained  neutral  parties  and  hostile  assets  that  did  not  directly  engage 
friendly  forces,  and  were  therefore  deemed  to  be  low  of  the  list  of  priorities  as  specified  by 
the  rules  of  engagement  and  commander’s  intent. 

3.  Results  and  Discussion 

The  primary  purposes  of  this  investigation  were  to:  1)  Pilot  test  the  methodologies  used  in 
the  experiment  (e.g.,  operationalization  of  the  two  organizational  structures;  training 
requirements);  and  2)  To  collect  preliminary  data  on  the  processes  that  underlie  performance 
for  these  organizations  coping  with  this  particular  mission.  Below,  we  present  our  findings  in 
the  context  of  lessons  learned  that  will  inform  the  design  of  subsequent  work. 

3.1  Measures  of  Performance 

Consideration  of  the  performance  of  the  teams  indicated  that  rather  than  performance  being 
superior  for  one  organizational  structure  over  another,  performance  depended  on  the  types  of 
tasks  to  which  the  teams  were  exposed.  For  instance.  Figure  2  shows  performance  measured 
as  the  number  of  attacks  attempted  divided  by  the  number  of  tasks  arrived  (present).  This 
measure  is  a  proportion  that  indicates  how  successful  the  teams  were  in  terms  of  their  ability 
to  process  the  tasks  that  were  present  and  that  needed  to  be  processed  (i.e.,  these  measures 
exclude  items  such  as  hostile  aircraft  that  were  not  threatening  and  therefore  did  not  need  to 
be  processed  by  the  players).  The  primary  strike  targets  included  performance  on  the  air  base 
and  naval  base,  which  were  identified  in  the  commander’s  intent  as  being  important  strike 
targets.  Time  critical  targets  (launchers)  included  performance  on  SCUD  launchers,  coastal 
defense  launchers,  enemy  destroyers,  and  enemy  submarines.  Each  of  these  tasks  threatened 
friendly  assets  or  geographical  areas  if  left  unprocessed.  Time  critical  targets  (missiles) 
included  SCUD  missiles,  coastal  defense  missiles,  exocet  missiles,  and  torpedoes  fired  by  the 
“launchers.”  Thus,  performance  on  time  critical  targets  (missiles)  represents  how  well  the 
teams  shot  down  missiles  if  they  were  unable  to  previously  destroy  the  launchers  shown  in 
the  time  critical  targets  (launchers)  columns.  These  data  indicated  that  performance  tended 
to  be  superior  in  the  functional  condition  for  the  primary  strike  targets  (paired  two-tailed  t  (6) 
=  -0.81,  ns)  and  the  launchers  (paired  two-tailed  t  (6)  =  -0.78,  ns),  whereas  the  divisional 
teams  outperformed  the  functional  teams  on  the  missiles  (paired  two-tailed  t  (6)  =  2.85,  p  < 
0.05).  Some  caution  should  be  taken  in  interpretation  of  these  data  because  these  trends  were 
small  and  did  not  reach  conventional  levels  of  significance.  However,  we  believe  that  these 
data  are  highly  suggestive  given  the  small  N  and  pilot  nature  of  the  investigation. 


Primary  Strike  Targets  Time  Critical  Targets  Time  Critical  Targets  (Missiles) 

(Launchers) 

Task  Type 

Figure  2.  Performance  of  teams  by  task  type,  where  performance  is  measured  as  the  number  of  attacks 
attempted  divided  by  the  number  of  tasks  arrived  (present). 

These  data  suggest  that  the  functional  teams  tended  to  perform  better  on  strike  targets,  but 
when  threatened,  the  divisional  teams  were  better  prepared  to  deal  with  defensive  actions. 
The  defensive  strength  of  the  divisional  teams  may  have  been  due  to  the  spread  of  defensive 
assets  across  multiple  players  (e.g.,  3  players  had  anti-ballistic  missiles),  whereas  in  the 
functional  condition,  defensive  assets  were  concentrated  in  single  players  (e.g.,  one  player 
had  all  of  the  anti-ballistic  missiles).  Since  the  SCUD  missiles  came  in  waves,  it  is  likely 
that  the  single  players  responsible  for  ballistic  missile  defense  in  the  functional  condition 
were  more  easily  overwhelmed.  In  contrast,  in  the  functional  condition  performance  on  the 
primary  strike  targets  may  have  tended  to  be  superior  in  the  functional  case  because  one 
player  could  concentrate  on  strike  and  another  could  concentrate  on  surveillance.  Both  of 
these  players  were  relatively  free  from  other  duties. 

In  fact,  these  data  are  consistent  with  observations  from  the  players.  Regarding  mission 
capabilities,  players  remarked  that: 

•  “In  functional,  because  we  had  to  rely  on  other  players  for  much  of  our  protection 
(instead  of  using  self-defense),  we  could  focus  more  on  offensive  mission 
responsibilities.” 

•  “In  divisional,  it  was  easier  to  do  self-defense  without  requiring  the  assistance  of 
other  players.” 


•  “Functional  had  better  allocation  of  ISR  because  these  assets  were  under  the  control 
of  one  individual.” 

•  “The  multi-function  capability  of  divisional  facilitated  load-shedding  if  workload 
became  too  high,  or  providing  back-up  or  assistance  if  others’  workload  became 
high.” 

•  “Divisional  had  a  greater  focus  on  self-defense.” 

Combined  with  the  performance  measures,  these  data  suggest  that  given  the  current  scenario, 
the  functional  teams  tended  to  have  a  slight  advantage  in  offensive  operations,  while  the 
divisional  teams  tended  to  have  a  slight  advantage  in  defensive  operations  because  of  the  way 
assets  were  allocated  and  the  pattern  of  their  use  (over  space  and  time)  during  the  mission. 

The  lessons  learned  here  are  two:  1)  Performance  needs  to  be  analyzed  by  consideration  of 
the  types  of  tasks  to  which  the  teams  were  exposed;  2)  Different  structural  organizations  may 
have  superior  capabilities  in  certain  areas  that  can  be  exploited,  thus  facilitating  the 
construction  of  scenarios  that  are  tuned  or  mismatched  to  organizational  proficiencies. 

3.2  Definition  of  Organizational  Structures 

As  noted  above,  in  general  the  organizational  structures  were  defined  such  that  in  the 
divisional  organization  players  had  control  of  a  multifunctional  platform,  whereas  in  the 
functional  organization  players  specialized  in  one  area  of  the  mission.  However,  this 
manipulation  was  not  entirely  “complete.”  For  instance,  in  the  divisional  organization,  each 
player  did  not  have  direct  control  of  assets  across  all  of  the  functional  areas.  Likewise,  in  the 
functional  organization,  although  players  had  primary  responsibility  in  one  functional  area, 
they  could  participate  in  modest  ways  in  other  areas  as  well.  For  example,  in  addition  to 
anti-ballistic  missiles,  the  player  in  the  role  of  ballistic  missile  defense  had  assets  (cruise 
missiles)  that  could  be  used  to  destroy  items  such  as  SCUD  launchers.  Yet,  these  cruise 
missiles  could  also  be  used  for  strike  against  items  such  as  the  air  base.  Similarly,  although 
the  player  in  the  role  of  strike  could  use  his  cruise  missiles  to  attack  items  such  as  the  air 
base,  these  same  assets  could  be  used  against  SCUD  launchers,  thereby  enabling  some 
ballistic  missile  defense.  This  means  that  our  organizations  were  not  purely  functional  or 
divisional,  but  rather,  they  were  somewhere  on  a  continuum  between  the  two  extremes. 

Moreover,  asset  distribution  was  not  the  only  thing  that  influenced  coordination 
requirements.  For  instance,  due  to  overlap  in  the  geographical  boundaries  between  players 
(given  asset  ranges),  players  operating  the  divisional  organization  had  to  coordinate  with 
each  other  over  boundaries  within  the  area  of  operations.  This  implies  that  while  the 
functional  condition  may  have  forced  coordination  across  functional  areas  (e.g.,  a  sensor  and 
a  strike  weapon  might  be  needed  to  hit  a  target),  the  divisional  condition  forced  coordination 
as  players  dealt  with  regional  boundaries.  Coordination  was  not  present  or  absent,  but 
instead,  coordination  requirements  were  different. 

Player’s  comments  reflected  this  difference  in  coordination  requirements,  and  were  often 
quite  contradictory  in  appearance.  For  instance,  regarding  coordination,  various  players 
remarked  that: 


•  “In  divisional,  there  was  more  autonomy  for  multi-function  tasks,  and  thus,  less  need 
to  coordinate.” 

•  “In  divisional,  we  needed  to  deconflict  with  other  players  capable  of  the  same  task.” 

•  “In  functional,  coordination  was  more  “contained”  in  that,  for  a  given  task,  we  only 
had  to  involve,  typically,  one  other  player  to  achieve  a  multifunctional  requirement. 
This  is  in  contrast  with  divisional  where  it  was  necessary  to  deconflict  with  several 
players  who  had  multi- function  capability.” 

•  “Because  functional  required  us  to  coordinate,  it  also  motivated  us  to  function  more 
as  a  “team”  than  in  divisional  where  we  operated  more  autonomously.” 

Note  that  in  related  points,  several  of  the  players  also  had  contradictory  remarks  about  how 
the  organizational  manipulation  affected  situational  awareness: 

•  “The  greater  requirement  for  coordination  in  functional  increased  the  situational 
awareness  in  comparison  with  divisional.” 

•  “Divisional  forced  the  players  to  continually  scan  all  aspects  of  the  battlespace  for 
opportunities  and  to  track  contacts,  thus  increasing  situational  awareness.” 

•  “Divisional  gave  each  player  a  more  limited  geographic  area  of  operation  to  focus  on 
so  situational  awareness  in  the  area  of  operation  was  stronger,  but  total  situational 
awareness  was  weaker.” 

•  “Because  in  functional  players  had  reach  across  the  entire  battlespace,  there  was 
greater  situational  awareness,  but  only  related  to  the  particular  functional  capability.” 

In  total,  then,  these  data  suggest  that  similar  to  coordination,  situational  awareness  was  not 
better  or  worse  across  organizations,  but  instead,  situational  awareness  was  different. 
Moreover,  these  data  suggest  that  situational  awareness  is  in  fact  very  difficult  to  precisely 
define. 

Collectively,  these  data  suggest  that  the  asset  distribution  across  players  and  the  demands  of 
the  mission  will  influence  both  how  much  and  what  type  of  coordination  is  necessary.  This 
is  a  critical  point  for  future  development,  for  the  “distance”  between  the  functional  and 
divisional  organizations  will  impact  the  “distance”  between  the  scenarios  that  are  tuned  to 
match  the  different  structures.  Since  the  goal  is  to  define  scenarios  that  selectively  challenge 
or  exploit  weaknesses  in  the  two  structures,  the  structures  need  to  be  different  enough  to 
generate  different  scenarios.  With  only  minor  differences  between  scenarios  and  structures, 
teams  will  be  likely  to  make  adaptations  that  are  not  structural  in  nature.  Thus,  for  our 
purposes,  this  distance  between  the  organizations  should  be  maximized  to  the  extent  possible 
(Figure  3). 

The  lessons  learned  here  are  two:  1)  Tight  definition  of  team  structures  is  necessary,  as  it  will 
likely  directly  affect  the  obtained  results;  and  2)  Players  coordinate  across  many  dimensions. 
Team  structure  alone  does  not  determine  coordination  requirements,  but  instead,  coordination 
requirements  are  subtly  determined  by  the  organizational  structure  operating  within  a 
particular  task  (mission)  domain. 


Figure  3.  An  abstract  space  representing  definitions  of  scenarios  that  either  fit  functional  (F)  or  divisional  (D) 
organizational  structures.  On  the  left  hand  side  of  the  figure,  the  F  and  D  are  spaced  far  apart,  such  that 
scenarios  tuned  to  the  organizations  are  substantially  dissimilar.  In  contrast,  on  the  right  hand  side  of  the  figure, 
the  F  and  D  are  spaced  close  together,  such  that  scenarios  tuned  to  the  organizations  are  substantially  similar. 
To  maximize  the  need  for  structural  adaptation,  the  stractures  need  to  be  spaced  far  apart.  Note  that  formal 
definition  of  these  spaces  depends  on  both  modeling  and  empirical  work. 

3.3  Structure  of  Training 

Two  issues  arise  in  a  consideration  of  training:  First,  how  much  training  should  be  provided? 
Second,  how  will  the  training  influence  subsequent  performance?  In  the  pilot  experiment 
reported  here,  participants  had  an  hour  of  training  prior  to  testing  in  each  organizational 
structure,  and  in  addition,  participants  conducted  “buttonology”  training  that  was  designed  to 
familiarize  them  with  the  functional  aspects  of  the  DDD.  During  this  “buttonology”  training, 
participants  launched  a  variety  of  assets  from  a  platform  under  their  control,  similar  to  the 
divisional  structure.  However,  players  were  also  able  to  launch  assets  off  of  a  different 
platform,  similar  to  the  functional  structure. 

The  issue  of  amount  of  training  is  critical  because  it  influences  the  stability  of  performance. 
Without  substantial  training,  it  is  possible  that  observations  will  reflect  how  players  learn  to 
play  the  DDD  rather  than  consistent  differences  between  organizational  and  mission 
structures.  However,  with  too  much  training,  it  is  possible  that  between  condition  differences 
may  disappear.  For  instance,  some  organizational  theorists  argue  that  teams  find  ways  to 
compensate  (e.g.,  leadership,  planning,  culture)  for  situations  when  structure  is  not  congruent 
with  the  mission  environment,  such  that  they  can  perform  as  well  as  organizations  that  have  a 
congruent  structure  (e.g.,  Donaldson,  2001).  Indeed,  one  player  remarked  that:  “Neither 
structure  should  be  viewed  as  more  capable  than  the  other  because,  while  each  may  have 
different  limitations,  these  limitations  can  be  overcome  in  different  ways.”  These  concerns 
suggest,  therefore,  that  one  needs  to  find  a  delicate  balance  between  training  too  little  and  too 
much.  Ideally,  time-series  based  measures  should  be  taken  as  “training”  evolves  so  that  one 
can  measure  how  teams  struggle  and  learn  to  adapt  to  their  mission  environments. 

The  issue  of  how  training  might  affect  performance  is  also  critical,  for  the  history  of  a  team’s 
activities  may  influence  current  performance.  For  instance.  Moon,  Hollenbeck,  Ilgen,  West, 
Ellis,  Humphrey,  and  Porter  (2000)  reported  that  performance  was  higher  for  teams  that 


transitioned  from  a  functional  to  a  divisional  structure  than  for  those  that  transitioned  from  a 
divisional  to  a  functional  structure.  These  data  suggest  that  team  performance  depends  on 
the  history  of  prior  activity,  and  that  training  may  begin  to  build  histories  that  constrain 
future  performance. 

Given,  then,  that  history  matters  -  that  team  behavior  is  path  dependent  -  it  is  likely  that 
much  more  is  being  learned  during  training  than  simply  how  to  play  the  DDD,  and  that  what 
is  being  learned  is  contributing  profoundly  to  the  results.  Simply  put,  training  needs  to  be 
carefully  considered,  for  even  slight  biases  in  training  (e.g.,  giving  a  team  more  experience  in 
one  organization  than  another,  or  the  order  of  that  training)  may  influence  results. 

Moreover,  the  effect  of  prior  experience  will  likely  interact  with  team  structure  and  mission 
structure.  Similar  to  the  congruence  effects  noted  above,  the  “distance”  between  team 
structures  will  influence  how  prior  experience  impacts  current  performance.  Figure  4  once 
again  shows  a  hypothetical  space  representing  definitions  of  scenarios  that  either  fit 
functional  (F)  or  divisional  (D).  In  the  middle  of  the  figure,  the  F  and  D  are  “spaced” 
moderately  apart,  whereas  on  the  right  hand  side  of  the  figure,  the  F  and  D  are  close  together. 
It  is  possible  that  due  to  similarity,  historical  effects  may  be  quite  robust  in  the  middle 
version,  in  which  the  scenarios  are  sufficiently  distinct,  yet  similar  enough,  to  interfere  with 
each  other.  In  contrast,  when  the  scenarios  are  highly  similar  as  on  the  right  side,  little 
interference  from  past  experience  may  occur  (i.e.,  little  adaptation  is  necessary).  Likewise, 
when  the  distance  between  the  F  and  D  is  large  as  on  the  left  side,  historical  effects  may  once 
again  be  small  as  the  similarity  decreases  (i.e.,  little  interference).  In  total,  the  magnitude, 
direction,  and  presence  of  order  effects  -  and  training  effects  -  may  depend  on  the  “distance” 
between  the  scenarios  (and  the  team  structures)  and  what  is  learned.  Of  course,  these  effects 
depend  on  whether  the  structures  and  scenarios  are  differentiated  along  meaningful  lines.  In 
fact,  the  presence  and  direction  of  these  effects  may  provide  information  about  the  structure 
of  the  space.  An  absence  of  a  training  or  order  effect  is  just  as  informative  as  the  presence  of 
such  an  effect. 


Figure  4.  An  abstract  space  representing  definitions  of  scenarios  that  either  fit  functional  (F)  or  divisional  (D) 
organizational  structures.  The  “distance”  between  the  F  and  D  will  likely  influence  the  observation  of  historical 
effects. 

Consistent  with  these  ideas,  data  from  the  pilot  experiment  suggest  that  the  history  of  prior 
activity  influenced  performance.  However,  this  influence  seemed  to  depend  on  task  type, 
consistent  with  the  data  shown  above  in  Figure  2.  Once  again.  Figure  5  shows  performance 
measured  as  the  number  of  attacks  attempted  divided  by  the  number  of  tasks  arrived  (present) 
for  the  primary  strike  targets  (top),  time  critical  targets  -  launchers  (middle),  and  time  critical 


targets  -  missiles  (bottom).  Here,  the  data  are  shown  by  trial  for  teams  who  went  from 
divisional  to  functional  (solid  line)  and  functional  to  divisional  (dashed  line).  For  the 
primary  strike  targets,  performance  in  the  functional  organization  tended  to  be  superior 
regardless  of  trial.  In  contrast,  for  the  missiles,  performance  in  the  divisional  organization 
tended  to  be  superior  regardless  of  trial.  However,  in  the  case  of  the  launchers,  performance 
was  superior  across  trials  in  the  divisional  to  functional  order.  This  means  that  divisional 
outperformed  functional  on  trial  1,  but  that  functional  outperformed  divisional  on  trial  2. 
These  data  suggest  that  order  effects  may  be  present  and  that  they  can  override  apparent 
advantages  for  functional  and  divisional  organizational  structures.  Note  that  some  caution  is 
need  in  interpreting  these  results,  given  that  these  effects  represent  only  trends.  However, 
given  that  these  data  are  based  on  a  low  N,  we  take  these  data  as  promising,  for  they  suggest 
that  history  may  matter  and  that  these  effects  may  depend  on  task  type. 

Here,  the  lessons-learned  are  three:  1)  Time-series  based  measures  of  performance  should  be 
taken  as  training  evolves  in  order  to  document  how  teams  struggle  to  adapt  to  their  missions; 

2)  Training  needs  to  be  carefully  crafted  so  as  to  be  of  a  proper  amount  and  non-biased;  and 

3)  The  presence  of  order  and  training  effects  will  depend  on  a  variety  of  factors,  including 
task  type. 

4.  Conclusions 

The  results  and  insights  gained  from  this  pilot  study  provide  valuable  guidance  as  we  move 
forward  toward  an  empirical  evaluation  of  organizational  congruence  and  adaptation.  The 
lessons  learned  suggest  empirical  strategies  such  as  analysis  by  task  type,  rigorous  attention 
to  the  consequences  of  team  structure  and  mission  definitions,  and  precise  planning  of  the 
types  and  amounts  of  training.  Each  of  these  factors  has  the  potential  to  profoundly  influence 
the  results,  reflecting  the  multiple  factors  that  shape  team  performance. 

In  particular,  the  preliminary  results  concerning  task  type  suggest  that  in  the  context  of  the 
scenario  studied  in  this  pilot  experiment,  the  functional  organization  was  less  proficient  at 
interception  of  waves  of  enemy  missiles.  As  noted  above,  this  may  reflect  the  fact  that,  for 
instance,  anti-ballistic  missiles  rested  in  the  hands  of  only  a  single  player  in  the  functional 
case.  Importantly,  this  preliminary  finding  suggests  a  strategy  for  designing  scenarios  that 
are  either  congruent  or  incongruent  with  team  structures:  Functional  organizations  may  be 
stressed  by  waves  of  attacks  focused  on  one  functional  area  (e.g.,  ballistic  missile  defense), 
whereas  divisional  organizations  may  be  stressed  by  waves  of  attacks  focused  on  one 
geographical  area  involving  assets  across  several  functional  areas  (i.e.,  thus  stressing  a  single 
divisional  region).  The  same  concept  -  swamping  of  a  particular  player  -  can  be 
accomplished  in  different  ways. 
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Figure  5.  Performance  by  task  type,  where  performance  is  measured  as  the  number  of  attacks  attempted 
divided  by  the  number  of  tasks  arrived  (present).  Data  are  shown  by  order,  where  divisional  to  functional 
is  the  solid  line  and  functional  to  divisional  is  the  dashed  line. 


Ultimately,  the  findings  reported  where  will  help  us  as  we  study  the  nature  of  congruence 
between  organizational  structures  and  missions,  and  as  we  strive  to  predict  and  measure  the 
effects  of  a  lack  of  congruence  on  the  behaviors  of  organizations  as  they  carry  out  missions. 
By  identifying  the  behaviors  of  organizations  coping  with  incongruence  -  by  understanding 
what  incongruence  looks  like  “in  action” —  we  hope  to  understand  how  organizations 
function  so  that  we  can  subsequently  understand  how  to  induce  radical  adaptation. 
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